feat!: migrate from prost to buffa for protobuf codegen - #557

Summary

Migrate protobuf codegen from prost/prost-build to buffa/buffa-build v0.8.1 (from crates.io), then build on top of buffa's zero-copy views to cut allocations on the hot receive/decode paths.

• Replace prost/prost-build with buffa/buffa-build (v0.8.1, from crates.io).
• Proto field names converted camelCase → snake_case (wire-compatible; preserves Rust field access patterns).
• Descriptor-based codegen: whatsapp.rs is no longer tracked — build.rs generates it into OUT_DIR from the committed whatsapp.desc (+ whatsapp.desc.sha256 freshness guard). Consumers never need protoc.
• Zero-copy views (generate_views=true) now used on real decode paths — not just enabled for the future.

API surface changes (codebase-wide)

Performance work on top of the migration

• Zero-copy views wired into signal-protocol decode, history-sync, app-state actions, SKDM-only plaintext, and the received-plaintext path.
• View-based session deserialize; fixed double compute_size traversal in session serialization.
• Unknown-field preservation disabled to drop allocation overhead.
• Session-record size cache (+ regression test).

Notable fixes / decisions

• is_sender_key_distribution_only: slow path encodes-and-compares, because buffa's MessageField equality treats set-to-default as equal to unset.
• whatsapp.rs removed from version control; build.rs fails fast on a stale descriptor or missing post-process marker.
• PollOption proto workaround dropped — buffa ≥ v0.7 handles the nested Option message, so the proto is back to upstream message Option.

Updated against main

This branch has been merged up to current main (through #683). The merge layered the buffa view architecture together with main's recent work, resolving conflicts by keeping the best of each side:

• history-sync (wacore/src/history_sync.rs): kept buffa's zero-copy view extraction as the base, and ported main's streaming decompression (perf: implement streaming decompression for history sync processing #672) and message-secret retention metadata — timestamp / is_poll_or_event / is_bot_invocation (feat(msg-secret)!: bound messageSecret retention by policy and event-time horizon #668) — onto it (prost-based extraction can't compile here).
• LazyHistorySync (events.rs): kept main's memory-freeing design (raw bytes freed after first decode, perf(history-sync): free LazyHistorySync raw bytes after a successful decode #669), adapted to buffa's decode_from_slice; dropped the buffa view() accessor (incompatible with freeing, no real consumers).
• device store (store/device.rs): combined main's Arc<…> wrapping (perf(send): Arc immutable device fields + recent-message bytes #674) with buffa's ADVSignedDeviceIdentity naming + MessageField API.
• messages (messages.rs): kept main's validate_bcl_hash / standard-base64 phash parity (fix(send): correct group phash and mark full SKDM target set (WA Web parity) #678/feat: WA Web phash parity — usync device_hash (#3), group-metadata phash (#7), bcl hash validation (#6) #679) + buffa's unpadded_message_len/unpad_message_ref split.
• Real bug fixed during the merge: the BCL-hash validation block in src/message.rs referenced the owned message before it was materialized on the buffa receive path — now reads the borrowed view.

Post-migration: binary size, perf regressions, publishability, descriptor guards

After the migration landed green, a measurement-driven cleanup pass:

• Binary size — buffa's generic codec methods are monomorphized in every calling crate and fat-LTO can't fold the (not byte-identical) copies, so the migration regressed .text by +1.83 MiB. Routing the hot decode/encode trees through #[inline(never)] non-generic entry points in waproto::codec (Message, WebMessageInfo, the history-sync records, and the send-path compute_size/write_to) collapses the per-crate duplicates into a single instantiation. Net .text regression is now ~+410 KiB (waproto-attributed bloat 3.04 → 1.59 MiB; llvm-lines wacore dropped below the prost baseline).
• Perf regressions — CodSpeed net +25% (17 improved, 4 regressed). The 4 remaining are inherent to buffa (owned-decode memory on the ~50-field SyncActionValue; the two-pass per-field encode scan on wide messages) and are documented; an earlier CPU regression in collect_unique_index_macs became a +28% improvement by decoding each index MAC once and byte-sorting instead of re-walking the MessageField chain in the sort comparator.
• Publishability — buffa deps moved from git/tag to crates.io 0.8.0, so cargo package / the release workflow no longer reject git-only dependencies.
• Descriptor freshness guards — the sqlite-storage wire.desc and voip MLow tables.desc build scripts now fail the build when the committed .desc no longer matches its .proto (the same sha256 guard waproto already had), so a stale descriptor can't silently generate code from an old schema.

Test plan

• cargo fmt --all — clean
• cargo sort -w — all Cargo.toml already sorted
• cargo clippy --all --tests — 0 warnings, 0 errors
• cargo test --workspace --exclude e2e-tests — 1,889 tests pass, 0 failures
• Binary-Size + CodSpeed CI measured (advisory): .text ~+410 KiB, perf net +25%
• Descriptor freshness guards verified: build fails on a tampered .proto, passes clean
• E2E tests (require mock server)

Known trade-off: closed enums drop unknown wire values

buffa generates proto2 enum fields as a closed Option<Enum> and drops any wire value outside the compiled schema at decode time (from_i32 -> None), where prost kept an open Option<i32> that round-tripped any integer. In practice WhatsApp only sends in-schema enum values today, so this is a forward-compatibility gap rather than an active bug, but it means an unknown enum value would vanish on a decode->re-encode path (own-device echo, history-sync persistence) and a hypothetical out-of-range SyncdMutation.operation would count as SET in an ltHash (recoverable via resync).

buffa already ships the open-enum runtime type (EnumValue<E> = Known(E)/Unknown(i32)), so the fix is an opt-in codegen flag rather than new machinery — tracked upstream at anthropics/buffa#269. We keep the closed-enum output for now and will switch the affected fields to EnumValue once that option lands, rather than hand-rolling int32 fields here and reverting the typed-enum ergonomics this PR is about.